Device and method for providing moving body information for a vehicle, and recording medium, on which a program for executing the method is recorded

ABSTRACT

A device for providing moving body information for a vehicle, according to one embodiment, includes an image acquisition unit acquiring a peripheral image of the vehicle, an image extraction unit extracting at least one of a first image of an upper body of at least one moving object or a second image of a lower body of the moving object from the peripheral image of the vehicle, and a moving object information determination unit determining at least one of presence/absence or a position of the moving object using at least one of the extracted first image or second image, wherein the image extraction unit extracts as the first image at least one of a ‘1-1’-st image of an entirety of the upper body or a ‘1-2’-nd image of a part of the upper body, and extracts as the second image at least one of a ‘2-1’-st image of an entirety of the lower body or a ‘2-2’-nd image of a part of the lower body.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is the National Phase of PCT International ApplicationNo. PCT/KR2016/007353, filed on Jul. 7, 2016, which claims priorityunder 35 U.S.C. 119(a) to Patent Application No. 10-2015-0097384, filedin the Republic of Korea on Jul. 8, 2015, all of which are herebyexpressly incorporated by reference into the present application.

TECHNICAL FIELD

Embodiments relate to a device and method for providing moving objectinformation for a vehicle, and a recording medium on which a program forexecuting the method is recorded.

BACKGROUND ART

Traffic accidents related to a person, particularly a child, present inthe vicinity of a vehicle due to a backward movement of a vehicle, haverecently become common. In order to prevent such traffic accidents,mounting the camera on the back of the vehicle is becoming mandatory inseveral countries. This is because installing a rear-view camera on avehicle can reduce traffic accidents by up to 25%, for example. Inparticular, when a rear-view camera is installed, a blind spot behindthe vehicle can be reduced compared to a case where a rear ultrasonicsensor is installed on the back of the vehicle. According to a studyconducted by the US Highway Safety Insurance Association in 2014, about6% of drivers having a rear ultrasonic sensor on their vehicle can avoidcollisions, while about 44% of drivers having a rear-view camera ontheir vehicle can avoid collisions.

Conventional rear-view cameras mounted on a vehicle cannot correctlyfigure out the presence or position of a person located in the vicinityof the vehicle due to various postures of a person located in thevicinity of the vehicle, hide of a part of the body, or distortion of afisheye lens. That is, information on a person located in the vicinityof the vehicle may not be accurately detected due to variouscircumstances regarding whether a person around the vehicle is standingor seated, whether the person's upper body is hidden or the person'slower body is hidden.

Although the method of part-based pedestrian detection is adopted inorder to cope with various environments around the vehicle as describedabove, this method results in a large amount of calculation in detectinga person.

DISCLOSURE Technical Problem

Embodiments provide a moving object information providing device andmethod for a vehicle which are capable of correctly providinginformation on a moving object located in the vicinity of a vehicle witha small amount of calculation, and a recording medium on which a programfor executing the method is recorded.

Technical Solution

In an embodiment, a moving object information providing device for avehicle may include an image acquisition unit configured to acquire aperipheral image of the vehicle, an image extraction unit configured toextract at least one of a first image of an upper body of at least onemoving object or a second image of a lower body of the moving objectfrom the peripheral image of the vehicle, and a moving objectinformation determination unit configured to determine at least one ofpresence/absence of the moving object around the vehicle or a positionof the moving object using at least one of the extracted first image orsecond image.

For example, the image extraction unit may extract, as the first image,at least one of a ‘1-1’-st image of an entirety of the upper body of themoving object or a ‘1-2’-nd image of a part of the upper body of themoving object, and extract, as the second image, at least one of a‘2-1’-st image of an entirety of the lower body of the moving object, ora ‘2-2’-nd image of a part of the lower body of the moving object.

For example, the image extraction unit may include at least one ‘1-1’-stimage detection unit configured to detect the ‘1-1’-st image from theacquired peripheral image, and at least one ‘2-1’-st image detectionunit configured to detect the ‘2-1’-st image from the acquiredperipheral image.

For example, the image extraction unit may include at least one of atleast one ‘1-1’-st image verification unit configured to verify the‘1-1’-st image detected by the ‘1-1’-st image detection unit, or atleast one ‘2-1’-st image verification unit configured to verify the‘2-1’-st image detected by the ‘2-1’-st image detection unit.

For example, the image extraction unit further may include at least onefirst 1-2 image detection unit configured to detect the ‘1-2’-nd imagefrom the ‘1-1’-st image detected by the ‘1-1’-st image detection unit orverified by the ‘1-1’-st image verification unit, and at least one‘2-2’-nd image detection unit configured to detect the ‘2-2’-nd imagefrom the ‘2-1’-st image detected by the ‘2-1’-st image detection unit orverified by the ‘2-1’-st image verification unit.

For example, the image extraction unit may include at least one of atleast one ‘1-2’-nd image verification unit configured to verify the‘1-2’-nd image detected by the ‘1-2’-nd image detection unit, or atleast one ‘2-2’-nd image verification unit configured to verify the‘2-2’-nd image detected by the ‘2-2’-nd image detection unit.

For example, the image extraction unit may extract at least one of thefirst or second image from the acquired peripheral image according tothe direction in which the moving object faces with respect to thevehicle.

For example, the at least one ‘1-1’-st image detection unit may includea ‘1-1-1’-st image detection unit configured to detect the ‘1-1’-stimage in a first direction in which the moving object faces the vehiclefrom the acquired peripheral image, and a ‘1-1-2’-nd image detectionunit configured to detect the ‘1-1’-st image in a second directiondifferent from the first direction from the acquired peripheral image,wherein the at least one ‘2-1’-st image detection unit may include a‘2-1-1’-st image detection unit configured to detect the ‘2-1’-st imagein the first direction from the acquired peripheral image, and a‘2-1-2’-nd image detection unit configured to detect the ‘2-1’-st imagein the second direction from the acquired peripheral image.

For example, the ‘1-1’-st image detection unit may include a firstcommon image detection unit configured to detect the ‘1-1’-st image fromthe acquired peripheral image, and a first image classification unitconfigured to classify the ‘1-1’-st image according to a direction inwhich the moving object faces with respect to the vehicle from the‘1-1’-st image detected by the first common image detection unit,wherein the ‘2-1’-st image detection unit may include a second commonimage detection unit configured to detect the ‘2-1’-st image from theacquired peripheral image, and a second image classification unitconfigured to classify the ‘2-1’-st image according to a direction inwhich the moving object faces with respect to the vehicle from the‘2-1’-st image detected by the second common image detection unit.

For example, the at least one ‘1-1’-st image verification unit mayinclude at least one of a ‘1-1-1’-st image verification unit configuredto verify the ‘1-1’-st image in the first direction detected by the‘1-1-1’-st image detection unit or classified by the first imageclassification unit, or a ‘1-1-2’-nd image verification unit configuredto verify the ‘1-1’-st image in the second direction detected by the‘1-1-2’-nd image detection unit or classified by the first imageclassification unit, wherein the at least one ‘2-1’-st imageverification unit may include at least one a ‘2-1-1’-st imageverification unit configured to verify the ‘2-1’-st image in the firstdirection detected by the ‘2-1-1’-st image detection unit or classifiedby the second image classification unit, or a ‘2-1-2’-nd imageverification unit configured to verify the ‘2-1’-st image in the seconddirection detected by the ‘2-1-2’-nd image detection unit or classifiedby the second image classification unit.

For example, the at least one ‘1-2’-nd image detection unit may includea ‘1-2-1’-st image detection unit configured to detect the ‘1-2’-ndimage in the first direction from the ‘1-1’-st image detected by the‘1-1-1’-st image detection unit, classified by the first imageclassification unit or verified by the ‘1-1-1’-st image verificationunit, and a ‘1-2-2’-nd image detection unit configured to detect the‘1-2’-nd image in the second direction from the ‘1-2’-nd image detectedby the ‘1-1-2’-nd image detection unit, classified by the first imageclassification unit or verified by the ‘1-1-2’-nd image verificationunit, wherein the at least one ‘2-2’-nd image detection unit further mayinclude a ‘2-2-1’-st image detection unit configured to detect the‘2-2’-nd image in the first direction from the ‘2-1’-st image detectedby the ‘2-1-1’-st image detection unit, classified by the second imageclassification unit or verified by the ‘2-1-1’-st image verificationunit, and a ‘2-2-2’-nd image detection unit configured to detect the‘2-2’-nd image in the second direction from the ‘2-1’-st image detectedby the ‘2-1-2’-nd image detection unit, classified by the second imageclassification unit or verified by the ‘2-1-2’-nd image verificationunit.

For example, the at least one first 1-2 image verification unit mayinclude at least one of a ‘1-2-1’-st image verification unit configuredto verify the ‘1-2’-nd image detected by the ‘1-2-1’-st image detectionunit, or a ‘1-2-2’-nd image verification unit configured to verify the‘1-2’-nd image detected by the ‘1-2-2’-nd image detection unit, whereinthe at least one ‘2-2’-nd image verification unit may include at leastone of a ‘2-2-1’-st image verification unit configured to verify the‘2-2’-nd image detected by the ‘2-2-1’-st image detection unit, or a‘2-2-2’-nd image verification unit configured to verify the ‘2-2’-ndimage detected by the ‘2-2-2’-nd image detection unit.

For example, the image extraction unit may include a ‘1-2’-nd imagedetection unit configured to detect the ‘1-2’-nd image from the acquiredperipheral image, and a ‘2-2’-nd image detection unit configured todetect the ‘2-2’-nd image from the acquired peripheral image.

For example, the image extraction unit further may include at least oneof a ‘1-2’-nd image verification unit configured to verify the ‘1-2’-ndimage detected by the ‘1-2’-nd image detection unit, or a ‘2-2’-nd imageverification unit configured to verify the ‘2-2’-nd image detected bythe ‘2-2’-nd image detection unit.

For example, the image extraction unit further may include a first imagedetermination unit configured to determine the ‘1-1’-st image from the‘1-2’-nd image detected by the ‘1-2’-nd image detection unit or verifiedby the ‘1-2’-nd image verification unit, and a second imagedetermination unit configured to determine the ‘2-1’-st image from the‘2-2’-nd image detected by the ‘2-2’-nd image detection unit or verifiedby the ‘2-2’-nd image verification unit.

For example, the image extraction unit further may include a ‘1-1’-stimage verification unit configured to verify the ‘1-1’-st imagedetermined by the first image determination unit, and a ‘2-1’-st imageverification unit configured to verify the ‘2-1’-st image determined bythe second image determination unit.

For example, the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd imageverification unit may use at least one of a position, a size, or apattern of the moving object to verify the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st,or ‘2-2’-nd image.

For example, each of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd imageverification units may include a third image verification unitconfigured to verify the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd imageusing the position and size of the moving object to identify the first,or a fourth image verification unit configured to recognize the patternof the moving object and verify the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or‘2-2’-nd image using the recognized result.

For example, each of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd imageverification units may include the third and fourth image verificationunits, wherein the fourth image verification unit may re-verify the‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image verified by the thirdimage verification unit.

For example, the image extraction unit may detect the first and secondimages from a detection area of the acquired peripheral image, whereinthe detection area may be an area of points within a first distance fromthe vehicle.

For example, the moving object information determination unit maydetermine at least one of presence/absence of the moving object or aposition of the moving object using an image extracted first among thefirst image and the second image.

For example, the moving object information providing device may furtherinclude a moving object alert unit configured to announce the presenceof the moving object in response to a result determined by the movingobject information determining unit, wherein the alert area may be anarea of points within a second distance from the vehicle.

For example, the moving object information determination unit mayinclude a position tracking unit configured to track the position of themoving object using at least one of the first or second image extractedby the image extraction unit, and an information generation unitconfigured to determine at least one of the presence/absence of themoving object or the position of the moving object using at least one ofthe extracted first image or second image, or the tracked position.

In another embodiment, a method for providing moving object informationfor a vehicle may include (a) acquiring a peripheral image of thevehicle, (b) extracting at least one of a first image of an upper bodyof at least one moving object or a second image of a lower body of themoving object from the acquired peripheral image, and (c) determining atleast one of presence/absence of the moving object or a position of themoving object using at least one of the extracted first image or secondimage.

For example, the first image may include at least one of a ‘1-1’-stimage of an entirety of the upper body of the moving object, or a‘1-2’-nd image of a part of the upper body of the moving object, whereinthe second image may include at least one of a ‘2-1’-st image of anentirety of the lower body of the moving object, or a ‘2-2’-nd image ofa part of the lower body of the moving object.

For example, the step (b) may include detecting the ‘1-1’-st image fromthe acquired peripheral image when the acquired peripheral imageincludes the ‘1-1’-st image, detecting the ‘2-1’-st image from theacquired peripheral image when the acquired peripheral image includesthe ‘2-1’-st image, verifying the detected ‘1-1’-st or ‘2-1’-st image,detecting the ‘1-2’-nd or ‘2-2’-nd image from the verified or detected‘1-1’-st or ‘2-1’-st image, respectively, and verifying the detected‘1-2’-nd or ‘2-2’-nd image.

For example, the detecting of the ‘1-1’-st image may include detectingthe ‘1-1’-st image of the first direction from the acquired peripheralimage when the moving object contained in the ‘1-1’-st image included inthe acquired first image faces in the first direction with respect tothe vehicle, and detecting the ‘1-1’-st image of a second directiondifferent from the first direction from the acquired peripheral imagewhen the moving object contained in the ‘2-1’-st image included in theacquired peripheral image faces in the second direction with respect tothe vehicle, wherein the detecting of the ‘2-1’-st image may includedetecting the ‘2-1’-st image of the first direction from the acquiredperipheral image when the moving object contained in the ‘2-1’-st imageincluded in the acquired second image faces in the first direction withrespect to the vehicle, and detecting the ‘2-1’-st image of the seconddirection from the acquired peripheral image when the moving objectcontained in the ‘2-1’-st image included in the acquired second imagefaces in the second direction with respect to the vehicle.

For example, the detecting of the ‘1-1’-st or ‘2-1’-st image may includedetecting the ‘1-1’-st image from the acquired peripheral image when the‘1-1’-st image is included in the acquired peripheral image, classifyingthe ‘1-1’-st image of a first direction from the detected ‘1-1’-st imagewhen the moving object contained in the detected ‘1-1’-st image faces inthe first direction with respect to the vehicle, classifying the‘1-1’-st image of a second direction different from the first directionfrom the detected ‘1-1’-st image when the moving object contained in thedetected ‘1-1’-st image faces in the second direction with respect tothe vehicle, detecting the ‘2-1’-st image from the acquired peripheralimage when the ‘2-1’-st image is included in the acquired peripheralimage, classifying the ‘2-1’-st image of the first direction from thedetected ‘2-1’-st image when the moving object contained in the detected‘2-1’-st image faces in the first direction with respect to the vehicle,and classifying the ‘1-1’-st image of the second direction differentfrom the first direction from the detected ‘2-1’-st image when themoving object contained in the detected ‘2-1’-st image faces in thesecond direction with respect to the vehicle.

For example, the step (b) may include detecting the ‘1-2’-nd image fromthe acquired peripheral image when the acquired peripheral imageincludes the ‘1-2’-nd image, detecting the ‘2-2’-nd image from theacquired peripheral image when the acquired peripheral image includesthe ‘2-2’-nd image, verifying the detected ‘1-2’-nd or ‘2-2’-nd image,determining the ‘1-1’-st or ‘2-1’-st image from the verified ‘1-2’-nd or‘2-2’-nd image, respectively, and verifying the determined ‘1-1’-st or‘2-1’-st image.

For example, the moving object information providing method may furtherinclude announcing at least one of presence/absence of the moving objector the position of the moving object.

For example, the step (c) may include-tracking the position of themoving object using at least one of the extracted first or second image.

For example, when the acquired peripheral image includes both the firstimage and the second image, only the second image may be extracted.

In another embodiment, a computer-readable recording medium on which aprogram for executing a moving object information providing method for avehicle performed by a moving object information providing device forthe vehicle is recorded may record a program configured to implement afunction (a) of acquiring a peripheral image of the vehicle, a function(b) of extracting at least one of a first image of an upper body of amoving object or a second image of a lower body of the moving objectfrom the acquired peripheral image, and a function (c) of determining atleast one of presence of the moving object or a position of the movingobject, using at least one of the extracted first or second image.

For example, in the program recorded on the computer-readable recordingmedium, the first image may include at least one of a ‘1-1’-st image ofan entirety of an upper body of the moving body or a ‘1-2’-nd image of apart of the upper body of the moving object, and the second image mayinclude at least one of a ‘2-1’-st image of an entirety of a lower bodyof the moving object or a ‘2-2’-nd image of a part of the lower body ofthe moving object.

For example, the function (b) implemented by the program recorded on thecomputer-readable recording medium may include a function of detectingthe ‘1-1’-st image from the acquired peripheral image when the acquiredperipheral image includes the ‘1-1’-st image, a function of detectingthe ‘2-1’-st image from the acquired peripheral image when the acquiredperipheral image includes the ‘2-1’-st image, a function of verifyingthe detected ‘1-1’-st or ‘2-1’-st image, a function of detecting the‘1-2’-nd or ‘2-2’-nd image from the verified ‘1-1’-st or ‘2-1’-st image,respectively, and a function of verifying the detected ‘1-2’-nd or‘2-2’-nd image.

For example, the function of detecting the ‘1-1’-st image, which isimplemented by the program recorded on the computer-readable recordingmedium, may include a function of detecting the ‘1-1’-st image of afirst direction from the acquired peripheral image when the movingobject contained in the ‘1-1’-st image included in the acquired firstimage faces in the first direction with respect to the vehicle, and afunction of detecting the ‘1-1’-st image of a second direction differentfrom the first direction from the acquired peripheral image when themoving object contained in the ‘2-1’-st image included in the acquiredperipheral image faces in the second direction with respect to thevehicle, wherein the function of detecting the ‘2-1’-st image mayinclude a function of detecting the ‘2-1’-st image of the firstdirection from the acquired peripheral image when the moving objectcontained in the ‘2-1’-st image included in the acquired peripheralimage faces in the first direction with respect to the vehicle, and afunction of detecting the ‘2-1’-st image of the second direction fromthe acquired peripheral image when the moving object contained in the‘2-1’-st image included in the acquired peripheral image faces in thesecond direction with respect to the vehicle.

For example, the function of detecting of the ‘1-1’-st or ‘2-1’-stimage, which is implemented by the program recorded on thecomputer-readable recording medium, may include a function of detectingthe ‘1-1’-st image from the acquired peripheral image when the ‘1-1’-stimage is included in the acquired peripheral image, a function ofclassifying the ‘1-1’-st image of a first direction from the detected‘1-1’-st image when the moving object contained in the detected ‘1-1’-stimage faces in the first direction with respect to the vehicle, afunction of classifying the ‘1-1’-st image of a second directiondifferent from the first direction from the detected ‘1-1’-st image whenthe moving object contained in the detected ‘1-1’-st image faces in thesecond direction with respect to the vehicle, a function of detectingthe ‘2-1’-st image from the acquired peripheral image when the ‘2-1’-stimage is included in the acquired peripheral image, a function ofclassifying the ‘2-1’-st image of the first direction from the detected‘2-1’-st image when the moving object contained in the detected ‘2-1’-stimage faces in the first direction with respect to the vehicle, and afunction of classifying the ‘2-1’-st image of the second direction fromthe first direction from the detected ‘2-1’-st image when the movingobject contained in the detected ‘2-1’-st image faces in the seconddirection with respect to the vehicle.

Advantageous Effects

According to an embodiment, a moving object information providing deviceand method for a vehicle and a recording medium on which a program forexecuting the method is recorded may accurately determine and provideinformation on a moving object with a small amount of calculation evenin a case where the moving object assumes various postures in thevicinity of the vehicle, a part of the moving object is hidden, or thefisheye lens causes distortion.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a moving object information providingdevice for a vehicle according to an embodiment.

FIG. 2 is a flowchart illustrating a method for providing moving objectinformation for a vehicle executed by the moving object informationproviding device for a vehicle shown in FIG. 1.

FIGS. 3A and 3B show an example of various images of surroundings forexplaining a moving object information providing device and method for avehicle according to an embodiment.

FIGS. 4A and 4B show another example of various images of surroundingsfor explaining a moving object information providing device and methodfor a vehicle according to an embodiment.

FIGS. 5A and 5B show yet another example of various images ofsurroundings for explaining a moving object information providing deviceand method for a vehicle according to an embodiment.

FIG. 6 is a block diagram of an embodiment of the image extraction unitshown in FIG. 1.

FIG. 7 is a flowchart illustrating an embodiment of step 220 of FIG. 2.

FIG. 8 is a block diagram of another embodiment of the image extractionunit shown in FIG. 1.

FIG. 9 is a flowchart illustrating another embodiment of step 220 ofFIG. 2.

FIG. 10 is a block diagram of another embodiment of the image extractionunit shown in FIG. 1.

FIG. 11 is a flowchart illustrating another embodiment of step 220 ofFIG. 2.

FIG. 12 is a block diagram of another embodiment of the image extractionunit shown in FIG. 1.

FIG. 13 is a flowchart illustrating another embodiment of step 220 ofFIG. 2.

FIG. 14 exemplarily shows a peripheral image for explaining verificationof a detected image.

FIG. 15 is a block diagram of an embodiment of the respective imageverification units shown in FIGS. 6, 8, 10, and 12.

FIG. 16 shows an exemplary peripheral image for explaining a detectionarea and an alert area.

FIG. 17 is a block diagram illustrating an embodiment of the movingobject information determination unit shown in FIG. 1.

BEST MODE

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the accompanying drawings in order tofacilitate understanding of the present disclosure. However, theembodiments according to the present disclosure can be modified invarious other forms, and the scope of the present disclosure should notbe construed as being limited to the embodiments described below. Theembodiments of the present disclosure are provided to enable thoseskilled in the art to more fully understand the present disclosure.

It is also to be understood that relational terms, such as first andsecond, on/upper portion/above and under/lower portion/below, are usedonly to distinguish between one subject or element and another subjector element without necessarily requiring or involving any physical orlogical relationship or sequence between such subjects or elements.

FIG. 1 is a block diagram of a moving object information providingdevice 100 for a vehicle according to an embodiment and may include animage acquisition unit 110, an image extraction unit 120, a movingobject information determination unit 130, and a moving object alarmingunit 140.

FIG. 2 is a flowchart illustrating a method 200 for providing movingobject information for a vehicle executed by the moving objectinformation providing device 100 for a vehicle shown in FIG. 1.

Hereinafter, the method 200 for providing moving object information fora vehicle illustrated in FIG. 2 is described as being executed by thedevice 100 shown in FIG. 1, but embodiments are not limited thereto.That is, the vehicle moving object information providing method 200shown in FIG. 2 may be executed by another device. While the movingobject information providing device 100 for a vehicle shown in FIG. 1 isdescribed as operating in the order shown in FIG. 2, embodiments are notlimited thereto. That is, according to another embodiment, the movingobject information providing device 100 shown in FIG. 1 may operate in adifferent order from the method 200 shown in FIG. 1.

Referring to FIGS. 1 and 2, the image acquisition unit 110 of the movingobject information providing device 100 according to the embodimentacquires a peripheral image of the vehicle and outputs the acquiredimage to the image extraction unit 120 (step 210). Here, the acquiredperipheral image may correspond to an image of at least one of thefront, rear, or side of the vehicle. For example, the image acquisitionunit 110 may be implemented in the form of a miniature camera that maybe mounted on a vehicle.

FIGS. 3A and 5B illustrate various images of surroundings for explainingthe device 100 and method 200 for providing moving object informationfor a vehicle according to an embodiment.

After step 210, the image extraction unit 120 may extract at least oneof a first image or a second image from the peripheral image of thevehicle (step 220).

Here, the “first image” means an image of an upper body of at least onemoving object, and the “second image” means an image of a lower body ofthe at least one moving object. For example, the first image means animage of an upper body I11 of a moving object MB assuming a squattingposition as shown in FIGS. 4A and 4B, and the second image is an imageof a lower body I21 of the moving object MB assuming a standing positionas shown in FIGS. 3A, 3B, 5A and 5B.

The term “moving object” means an object located in the vicinity of thevehicle, which means both a stationary but movable object in thevicinity of the vehicle and an object moving around the vehicle. Forexample, the moving object may be a person squatting around the vehicle,a pedestrian passing around the vehicle, a person standing around thevehicle, a person riding a bike, and the like, but the embodiment is notlimited to a specific type of the moving object. For example, for safetyof vehicle operation, any object that the driver or occupant of thevehicle needs to know about may correspond to the moving object.

In addition, the number of moving objects contained in the peripheralimage may be one or more.

The first image may include at least one of a ‘1-1’-st image or a‘1-2’-nd image. Here, the ‘1-1’-st image may mean an image of the wholeupper body of the moving object, and the ‘1-2’-nd may mean an image of apart of the upper body of the moving object. For example, when themoving object is a person, the ‘1-1’-st image may mean an image of theupper body of the person, and the ‘1-2’-nd image may mean an image of apart of the upper body of the person. For example, referring to FIGS. 4Aand 4B, the ‘1-1’-st image I11 may be an image of an upper body of aperson MB, and the ‘1-2’-nd image I12 may mean an image of the head ofthe person, which is a part of the upper body of the person. The imageextraction unit 120 may extract at least one of the ‘1-1’-st image orthe ‘1-2’-nd image as a first image.

A second image may include at least one of a ‘2-1’-st image or a‘2-2’-nd image. Here, the ‘2-1’-st image may mean an image of the wholelower body of the moving object, and the ‘2-2’-nd image may mean animage of a part of the lower body of the moving object. For example,when the moving object is a person, the ‘2-1’-st image may mean an imageof the lower body of the person, and the ‘2-2’-nd image may mean animage of a part of the lower body of the person. For example, referringto FIGS. 3A, 3B, 5A and 5B, the ‘2-1’-st image I21 may be an image ofthe lower body of a person MB, and the ‘2-2’-nd image I22-1, I22-2 maymean an image of feet, which is a part of the lower body of the person.Here, while it is illustrated that the number of the second ‘2-2’-ndimages I22-1 and I22-2 is two, a single ‘2-2’-nd image or more than two‘2-2’-nd images may be provided. The image extraction unit 120 mayextract at least one of the ‘2-1’-st image or the ‘2-2’-nd images as thesecond image.

The peripheral image acquired by the image acquisition unit 110 mayinclude at least one of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-ndimage. For example, the peripheral image may include only the ‘2-1’-stimage I21 and the ‘2-2’-nd images I22-1 and I22-2 and may not includethe ‘1-1’-st and ‘1-2’-nd images, as shown in FIGS. 3A and 3B, or mayinclude the ‘1-1’-st image, the ‘1-2’-nd image, ‘2-1’-st image I21, andthe ‘2-2’-nd image I22-1 and I22-2, as shown in FIGS. 5A and 5B.

Hereinafter, the embodiments of the image extraction unit 120 shown inFIG. 1 and step 220 shown in FIG. 2 will be described with reference tothe accompanying drawings.

FIG. 6 is a block diagram of an embodiment 120A of the image extractionunit 120 shown in FIG. 1, which may include a ‘1-1’-st image detectionunit 122A, a ‘1-1’-st image verification unit 124A, a ‘1-2’-nd imagedetection unit 126A, and a ‘1-2’-nd image verification unit 128A, a‘2-1’-st image detection unit 122B, a ‘2-1’-st image verification unit124B, a ‘2-2’-nd image detection unit 126B, and a ‘2-2’-nd imageverification unit 128B.

FIG. 7 is a flowchart illustrating an embodiment 220A of step 220 shownin FIG. 2.

Hereinafter, step 220A shown in FIG. 7 is described as being performedby the image extraction unit 120A shown in FIG. 6, but embodiments arenot limited thereto. That is, step 220A shown in FIG. 7 may be performedby the image extraction unit 120 having a different configuration fromthe image extraction unit 120A shown in FIG. 6. The image extractionunit 120A shown in FIG. 6 is described as operating in the order shownin FIG. 7, but embodiments are not limited thereto. That is, accordingto another embodiment, the image extraction unit 120A shown in FIG. 6may operate in a different order than in the method illustrated in FIG.7.

First, it is determined whether an peripheral image acquired by theimage acquisition unit 110 includes a ‘2-1’-st image (step 302). If theperipheral image includes the ‘2-1’-st image, the ‘2-1’-st imagedetection unit 122B may detect the ‘2-1’-st image from the peripheralimage received from the image acquisition unit 110 through an inputterminal IN1 (step 314).

Operations 302 and 314 may be performed by the ‘2-1’-st image detectionunit 122B for the following reason. If the ‘2-1’-st image is included inthe peripheral image, the ‘2-1’-st image may be detected by the ‘2-1’-stimage detection unit 122B. If the ‘2-1’-st image is not included in theperipheral image, the ‘2-1’-st image detection unit 122B may not detectthe ‘2-1’-st image.

If the peripheral image does not include the ‘2-1’-st image, it isdetermined whether the peripheral image includes the ‘1-1’-st image(step 304). If the peripheral image includes the ‘1-1’-st image, the‘1-1’-st image detection unit 122A detects the ‘1-1’-st image from theperipheral image received from the image acquisition unit 110 via theinput terminal IN1 (step 306).

Operations 302, 304, and 306 may be performed by the ‘1-1’-st imagedetection unit 122A for the following reason. If the ‘1-1’-st image isincluded in the peripheral image, the ‘1-1’-st image may be detected bythe ‘1-1’-st image detection unit 122A. If the ‘1-1’-st image is notincluded in the peripheral image, the ‘1-1’-st image may not be detectedby the ‘1-1’-st image detection unit 122A.

If the peripheral image includes both the ‘1-1’-st image and the‘2-1’-st image according to the situation in the vicinity of thevehicle, that is, if the upper and lower bodies of the moving objectlocated in the vicinity of the vehicle are all contained in theperipheral image, the image extraction unit 120, 120A may extract atleast one of a first image related to the upper body or a second imagerelated to the lower body. The embodiment 220A shown in FIG. 7 detectsonly the second image without detecting the first image, even if boththe first image and the second image are contained in the peripheralimage. For example, as shown in FIGS. 5A and 5B, when both the firstimage and the second image are contained in the peripheral image, theimage extraction unit 120A may detect only the second image I21.However, embodiments are not limited thereto.

According to another embodiment, when both the first image and thesecond image are contained in the peripheral image, the image extractionunit 120A may detect only the first image without detecting the secondimage. For example, as illustrated in FIGS. 4A and 4B, when the movingobject MB is squatting, only the first image I11 may be detected in theperipheral image because the lower body is hidden.

According to another embodiment, when both the first image and thesecond image are contained in the peripheral image, the image extractionunit 120A may detect both the first image and the second image.

After step 306, the ‘1-1’-st image verification unit 124A may verify the‘1-1’-st image detected by the ‘1-1’-st image detection unit 122A, andoutput the verified result to the ‘1-2’-nd image detection unit 126A(step 308). In addition, after step 314, the ‘2-1’-st image verificationunit 124B may verify the ‘2-1’-st image detected by the ‘2-1’-st imagedetection unit 122B and output the verified result to the ‘2-2’-nd imagedetection unit 126B (step 316).

After step 308, the ‘1-2’-nd image detection unit 126A detects the‘1-2’-nd image from the ‘1-1’-st image that is verified by the ‘1-1’-stimage verification unit 124A (step 310). In addition, after step 316,the ‘2-2’-nd image detection unit 126B detects the ‘2-2’-nd image fromthe ‘2-1’-st image verified by the ‘2-1’-st image verification unit 124B(step 318).

At least one of the ‘1-1’-st image verification unit 124A or the‘2-1’-st image verification unit 124B shown in FIG. 6 and at least oneof step 308 or step 316 shown in FIG. may be omitted. In the case wherethe ‘1-1’-st image verification unit 124A and step 308 are omitted,after step 306, the ‘1-2’-nd image detection unit 126A detects the‘1-2’-nd image from the ‘1-1’-st image detected by the ‘1-1’-st imagedetection unit 122A (step 310). In the case where the ‘2-1’-st imageverification unit 124B and step 316 are omitted, after step 314, the‘2-2’-nd image detection unit 126B detects the ‘2-2’-nd image from the‘2-1’-st image detected by the ‘2-1’-st image detection unit 122B (step318).

After step 310, the ‘1-2’-nd image verification unit 128A may verify the‘1-2’-nd image detected by the ‘1-2’-nd image detection unit 126A, andoutput the verified result to the moving object informationdetermination unit 130 via an output terminal OUT11. (step 312). Afterstep 318, the ‘2-2’-nd image verification unit 128B verifies the‘2-2’-nd image detected by the ‘2-2’-nd image detection unit 126B, andoutputs the verified result to the moving object informationdetermination unit 130 via an output terminal OUT21 (step 320).

According to another embodiment, at least one of the first ‘1-2’-ndimage verification unit 128A or the second ‘2-2’-nd image verificationunit 128B shown in FIG. 6 may be omitted, and at least one of step 312or step 320 shown in FIG. 7 may be omitted.

According to another embodiment, the ‘1-1’-st image detection unit 122A,the ‘1-1’-st image verification unit 124A, the ‘2-1’-st image detectionunit 122B, and the ‘2-1’-st image verification unit 124B shown in FIG. 6may be omitted, and steps 306, 308, 314, and 316 shown in FIG. 7 may beomitted. In this case, the ‘1-2’-nd image detection unit 126A may detectthe ‘1-2’-nd image from the peripheral image provided via the inputterminal IN1, and the ‘2-2’-nd image detection unit 126B may detect the‘2-2’-nd image from the peripheral image provided via the input terminalIN1. The first ‘1-2’-nd image verification unit 128A and the second‘2-2’-nd image verification unit 128B operate as described above.Therefore, the image extraction unit 120A may not provide the ‘1-1’stand ‘2-1’-st images but may provide at least one of the ‘1-2’-nd or‘2-2’-nd image to the moving object information determination unit 130.

According to another embodiment, the first ‘1-2’-nd image detection unit126A, the ‘1-2’-nd image verification unit 128A, the ‘2-2’-nd imagedetection unit 126B, and the ‘2-2’-nd image verification unit 128B shownin FIG. 6 may be omitted, and steps 310, 312, 318, and 320 shown in FIG.7 may be omitted. In this case, the first ‘1-1’-st image verified by the‘1-1’-st image verification unit 124A may be output to the moving objectinformation determination unit 130 via the output terminal OUT11, andthe ‘2-1’-st image verified by the ‘2-1’-st image verification unit 124Bmay be output to the moving object information determination unit 130via the output terminal OUT21. Therefore, the image extraction unit 120Amay not provide the ‘1-2’-nd and ‘2-2’-nd images but may provide atleast one of the ‘1-1’st or ‘2-1’-st image to the moving objectinformation determination unit 130.

There are various methods of detecting the corresponding images in the‘1-1’-st image detection unit 122A, the ‘1-2’-nd image detection unit126A, the ‘2-1’-st image detection unit 122B, and the ‘2-2’-nd imagedetection unit 126B. For example, a formatted pattern of each of the‘1-1’st, ‘1-2’-nd, ‘2-1’-st, and ‘2-2’-nd images may be pre-created andstored for each type of moving object, and the stored pattern may becompared with the pattern of the peripheral image to detect the ‘1-1’st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image from the peripheral image.

Generally, the pattern of the ‘1-1’-st image is larger than the patternof the ‘1-2’-nd image, and the pattern of the ‘2-1’-st image is largerthan the pattern of the ‘2-2’-nd image. If the ‘1-2’-nd image (or the‘2-2’-nd image) is detected directly from the peripheral image ratherthan from the ‘1-1’-st image (or the ‘2-1’-st image), the small patternof the ‘1-2’-nd image (or the ‘2-2’-nd image) needs to be directlycompared with the pattern of the peripheral image, and thus the amountof computation may be increased. However, in the embodiment shown inFIGS. 6 and 7, the pattern of the ‘1-2’-nd image (or the ‘2-2’-nd image)is not directly compared with the peripheral image. Instead, the‘1-1’-st image (or the ‘2-1’-st image) is first detected by comparingthe pattern of the ‘1-1’-st image (or the ‘2-1’-st image) with theperipheral image, and then the pattern of the ‘1-2’-nd image (or the‘2-2’-nd image) is compared with the pattern of the ‘1-1’-st image (orthe ‘2-1’-st image), which is smaller than the pattern of the peripheralimage. Therefore, the amount of computation needed to detect the‘1-2’-nd image (or the ‘2-2’-nd image) may be reduced.

FIG. 8 is a block diagram of another embodiment 120B of the imageextraction unit 120 shown in FIG. 1, which may include ‘1-1-1’-st and‘1-1-2’-nd image detection units 122A-1 and 122A-2, ‘1-1-1’-st and‘1-1-2’-nd image verification units 124A-1 and 124A-2, ‘1-2-1’-st and‘1-2-2’-nd image detection units 126A-1 and 126A-2, ‘1-2-1’-st and‘1-2-2’-nd image verification units 128A-1 and 128A-2, ‘2-1-1’-st and‘2-1-2’-nd image detection units 122B-1 and 122B-2, ‘2-1-1’-st and‘2-1-2’-nd image verification units 124B-1 and 124B-2, ‘2-2-1’-st and‘2-2-2’-nd image detecting units 126B-1 and 126B-2, and ‘2-2-1’-st and‘2-2-2’-nd image verification units 128B-1, 128B-2.

FIG. 9 is a flowchart illustrating another embodiment 220B of step 220shown in FIG. 2.

Hereinafter, step 220B shown in FIG. 9 is described as being performedby the image extraction unit 120B shown in FIG. 8, but embodiments arenot limited thereto. That is, step 220B shown in FIG. 9 may be performedby the image extraction unit 120 having a different configuration fromthe image extraction unit 120B shown in FIG. 8. The image extractionunit 120B shown in FIG. 8 is described as operating in the order shownin FIG. 9, but embodiments are not limited thereto. That is, accordingto another embodiment, the image extraction unit 120B shown in FIG. 8may operate in a different order from the method 220B shown in FIG. 9.

The image extraction unit 120B shown in FIG. 8 may receive an peripheralimage acquired by the image acquisition unit 110 via the input terminalIN1, and extract from the acquired peripheral image at least one of afirst image or a second image for each direction in which the movingobject faces with respect to the vehicle. For simplicity, the directionin which the front surface of the moving object faces the vehicle isdefined as a first direction, and a direction different from the firstdirection is defined as a second direction. Accordingly, when the movingobject faces the vehicle, that is, when the moving object faces in thefirst direction, the peripheral image includes a front image of themoving object. When the moving object faces away from the vehicle or aside of the moving object faces the vehicle, that is, when the movingobject faces in the second direction, the peripheral image may include aside image or rear image of the moving object. Thus, the imageextraction unit 120B may extract at least one of the first image or thesecond image for each direction in which the moving object faces.

First, it is determined whether the peripheral image obtained by theimage acquisition unit 110 may include a ‘2-1’-st image (step 402). Ifthe peripheral image includes the ‘2-1’-st image, it is determinedwhether the moving object faces in the first direction (step 408). Ifthe moving object faces in the first direction, the ‘2-1-1’-st imagedetection unit 122B-1 detects the ‘2-1’-st image in the first directionfrom the peripheral image received from the image acquisition unit 110via the input terminal IN1 (step 432).

Steps 402, 408, and 432 are performed by the ‘2-1-1’-st image detectionunit 122B-1 for the following reason. If the peripheral image includesthe ‘2-1’-st image of the whole lower body of the moving object facingin the first direction, the ‘2-1-1’-st image detection unit 122B-1 maydetect the ‘2-1’-st image in the first direction. However, if the‘2-1’-st image in the first direction is not included in the peripheralimage, the ‘2-1-1’-st image detection unit 122B-1 cannot detect the‘2-1’-st image in the first direction.

On the other hand, if the peripheral image includes the ‘2-1’-st imagebut the moving object faces in the second direction rather than in thefirst direction, the ‘2-1-2’-nd image detection unit 122B-2 detects the‘2-1’ image in the second direction different from the first directionfrom the peripheral image received from the acquisition unit 110 via theinput terminal IN1 (Step 442).

Steps 402, 408, and 442 are performed by the ‘2-1-2’-nd image detectionunit 122B-2 for the following reason. If the peripheral image includesthe ‘2-1’-st image of the whole lower body of the moving object facingin the second direction, the ‘2-1-2’-nd image detection unit 122B-2 maydetect the ‘2-1’-st image in the second direction. However, if the‘2-1’-st image in the second direction is not included in the peripheralimage, the ‘2-1-2’-nd image detection unit 122B-2 cannot detect the‘2-1’-st image in the second direction.

If the peripheral image does not include the ‘2-1’-st image, it isdetermined whether the peripheral image includes the ‘1-1’-st image(step 404). If the peripheral image includes the ‘1-1’-st image, it isdetermined whether the moving object faces in the first direction (step406). If the moving object faces in the first direction, the ‘1-1-1’-stimage detection unit 122A-1 detects the ‘1-1’-st image in the firstdirection from the peripheral image received from the image acquisitionunit 110 via the input terminal IN1 (step 412).

Steps 402, 404, 406, and 412 are performed by the ‘1-1-1’-st imagedetection unit 122A-1 for the following reason. If the ‘1-1’-st image ofthe whole upper body of the moving object facing in the first directionis included in the peripheral image, the ‘1-1-1’-st image detection unit122A-1 may detect the ‘1-1’-st image in the first direction. However, ifthe ‘1-1’-st image in the first direction is not included in theperipheral image, the ‘1-1-1’-st image detection unit 122A-1 cannotdetect the ‘1-1’-st image in the first direction.

On the other hand, if the peripheral image includes the ‘1-1’-st imagebut the moving object faces in the second direction different from thefirst direction, the ‘1-1-2’-nd image detection unit 122A-2 detects the‘1-1’ image in the second direction from the peripheral image receivedfrom the first image sensor 110 via the input terminal IN1 (step 422).

Steps 402, 404, 406, and 422 are performed by the ‘1-1-2’-nd imagedetection unit 122A-2 for the following reason. If the ‘1-1’-st image ofthe whole upper body of the moving object facing in the second directionis included in the peripheral image, the ‘1-1-2’-nd image detection unit122A-2 may detect the ‘1-1’-st image in the second direction. However,if the ‘1-1’-st image in the second direction is not included in theperipheral image, the ‘1-1-2’-nd image detection unit 122A-2 cannotdetect the ‘1-1’-st image in the second direction.

While it is illustrated in step 220B shown in FIG. 9 that the firstimage is not detected and only the second image is detected when boththe first image and the second image are contained in the peripheralimage, embodiments are not limited thereto. That is, according toanother embodiment, when both the first image and the second image arecontained in the peripheral image, only the first image may be detectedwithout detecting the second image. According to another embodiment,when both the first image and the second image are contained in theperipheral image, both the first image and the second image may bedetected.

After step 412, the ‘1-1-1’-st image verification unit 124A-1 may verifythe ‘1-1’-st image in the first direction as detected by the ‘1-1-1’-stimage detection unit 122A-1, and output a verified result to the‘1-2-1’-st image detection unit 126A-1 (step 414). After step 432, the‘2-1-1’-st image verification unit 124B-1 may verify the ‘2-1’-st imagein the first direction as detected by the ‘2-1-1’-st image detectionunit 122B-1 and output a verified result to the ‘2-2-1’-st imagedetection unit 126B-1 (step 434).

After step 422, the ‘1-1-2’-nd image verification unit 124A-2 may verifythe ‘1-1’-st image in the second direction as detected by the ‘1-1-2’-ndimage detection unit 122A-2 and output a verified result to the‘1-2-2’-nd image detection unit 126A-2 (step 424).

After step 442, the ‘2-1-2’-nd image verification unit 124B-2 may verifythe ‘2-1’-st image in the second direction as detected by the ‘2-1-2’-ndimage detection unit 122B-2 and output a verified result to the‘2-2-2’-nd image detection unit 126B-2 (step 444).

After step 414, the ‘1-2-1’-st image detection unit 126A-1 detects the‘1-2’-nd image in the first direction from the ‘1-1’-st image in thefirst direction, which is verified by the ‘1-1-1’-st image verificationunit 124A-1 (step 416).

After step 424, the ‘1-2-2’-nd image detection unit 126A-2 detects the‘1-2’-nd image in the second direction from the ‘1-1’-st image in thesecond direction, which is verified by the ‘1-1-2’-nd image verificationunit 124A-2, (step 426).

After step 434, the ‘2-2-1’-st image detection unit 126B-1 may detectthe ‘2-2’-nd image in the first direction from the ‘2-1’-st image in thefirst direction, which is verified by the ‘2-1-1’-st image verificationunit 124B-1, and output the verified ‘2-2’-nd image in the firstdirection to the ‘2-2-1’-st image verification unit 128B-1 (step 436).

After step 444, the ‘2-2-2’-nd image detection unit 126B-2 detects the‘2-2’-nd image in the second direction from the ‘2-1’-st image in thesecond direction, which is verified by the ‘2-1-2’-nd image verificationunit 124B-2, and outputs the detected image to the ‘2-2-2’-nd imageverification unit 128B-2 (step 446).

In some cases, at least one of the ‘1-1-1’-st, ‘1-1-2’-nd, ‘2-1-1’-st or‘2-1-2’-nd image verification unit 124A-1, 124A-2, 124B-1, or 124B-2shown in FIG. 8 may be omitted, and at least one of step 414, 424, 434,or 444 shown in FIG. 9 may be omitted.

If the ‘1-1-1’-st image verification unit 124A-1 and step 414 areomitted, the ‘1-2-1’-st image detection unit 126A-1 may detect the‘1-2’-nd image in the first direction from the ‘1-1’-st image in thefirst direction as detected by the ‘1-1-1’-st image detection unit122A-1, and output the detected ‘1-2’-nd image in the first direction tothe ‘1-2-1’-st image detection unit 128A-1.

If the ‘1-1-2’-nd image verification unit 124A-2 and step 424 areomitted, the ‘1-2-2’-nd image detection unit 126A-2 may detect the‘1-2’-nd image in the second direction from the ‘1-1’-st image in thesecond direction as detected by the ‘1-1-2’-nd image detection unit122A-2 and output the detected ‘1-2’-nd image in the second direction tothe ‘1-2-2’-nd image detection unit 128A-2.

If the ‘2-1-1’-st image verification unit 124B-1 and step 434 areomitted, the ‘2-2-1’-st image detection unit 126B-1 may detect the‘2-2’-nd image in the first direction from the ‘2-1’-st image in thefirst direction as detected by the ‘2-1-1’-st image detection unit122B-1, and output the detected ‘2-2’-nd image in the first direction tothe ‘2-2-1’-st image verification unit 128B-1.

If the ‘2-1-2’-nd image verification unit 124B-2 and step 444 areomitted, the ‘2-2-2’-nd image detection unit 126B-2 may detect the‘2-2’-nd image in the second direction from the ‘2-1’-st image in thesecond direction as detected by the ‘2-1-2’-nd image detection unit122B-2, and output the detected ‘2-2’-nd image in the second directionto the ‘2-2-2’-nd image verification unit 128B-2.

According to another embodiment, the ‘1-1-1’-st image detection unit122A-1, the ‘1-1-1’-st image verification unit 124A-1, the ‘1-1-2’-ndimage detection unit 122A-2, the ‘1-1-2’-nd image verification unit124A-2, the ‘2-1-1’-st image detection unit 122B-1, the ‘2-1-1’-st imageverification unit 124B-1, the ‘2-1-2’-nd image detection unit 122B-2 andthe ‘2-1-2’-nd image verification unit 124B-2 shown in FIG. 8 may beomitted. That is, steps 412, 414, 422, 424, 432, 434, 442 and 444 shownin FIG. 9 may be omitted. In this case, the ‘1-2-1’-st image detectionunit 126A-1 may detect the ‘1-2’-nd image in the first direction fromthe peripheral image provided via the input terminal IN1 (step 416), andthe ‘1-2-2’-nd image detection unit 126A-2 may detect the ‘1-2’-nd imagein the second direction from the peripheral image provided via the inputterminal IN1 (step 426). In addition, the ‘2-2-1’-st image detectionunit 126B-1 may detect the ‘2-2’-nd image in the first direction fromthe peripheral image provided via the input terminal IN1 (step 436), andthe ‘2-2-2’-nd image detection unit 126B-2 may detect the ‘2-2’-nd imagein the second direction from the peripheral image provided via the inputterminal IN1 (step 446). Therefore, the image extraction unit 120B maynot provide the ‘1-1’-st and ‘2-1’-st images, but may provide at leastone of the ‘1-2’-nd or ‘2-2’-nd image in the first or second directionto the moving object information determination unit 130.

According to another embodiment, the ‘1-2-1’-st image detection unit126A-1, the ‘1-2-1’-st image verification unit 128A-1, the ‘1-2-2’-ndimage detection unit 126A-2, the ‘1-2-2’-nd image verification unit128A-2, the ‘2-2-1’-st image detection unit 126B-1, the ‘2-2-1’-st imageverification unit 128B-1, the ‘2-2-2’-nd image detection unit 126B-2 andthe ‘2-2-2’-nd image verification unit 128B-2 shown in FIG. 8 may beomitted. That is, steps 416, 418, 426, 428, 436, 438, 446, and 448 shownin FIG. 9 may be omitted. In this case, the ‘1-1’-st image in the firstdirection, which is verified by the ‘1-1-1’-st image verification unit124A-1, may be output to the moving object information determinationunit 130 via the output terminal OUT111, and the ‘1-1’-st image in thesecond direction, which is verified by the ‘1-1-2’-nd image verificationunit 124A-2, may be output to the moving object informationdetermination unit 130 via the output terminal OUT112. The ‘2-1’-stimage in the first direction, which is verified by the ‘2-1-1’-st imageverification unit 124B-1, may be output to the moving object informationdetermination unit 130 via the output terminal OUT211, and the ‘2-1’-stimage in the second direction, which is verified by the ‘2-1-2’-nd imageverification unit 124B-2, may be output to the moving object informationdetermination unit 130 via the output terminal OUT212. Therefore, theimage extraction unit 120B may not provide the ‘1-2’-nd image and the‘2-2’-nd image, but may provide at least one of the ‘1-1’-st or ‘2-1’-stimage in the first or second direction to the moving object informationdetermination unit 130.

After step 416, the ‘1-2-1’-st image verification unit 128A-1 may verifythe ‘1-2’-nd image in the first direction detected by the ‘1-2-1’-stimage detection unit 126A-1, and output the verified result to themoving object information determination unit 130 via the output terminalOUT111 (step 418).

After step 426, the ‘1-2-2’-nd image verification unit 128A-2 may verifythe ‘1-2’-nd image in the second direction detected by the ‘1-2-2’-ndimage detection unit 126A-2, and output the verified result to themoving object information determination unit 130 via the output terminalOUT112 (step 428).

After step 436, the ‘2-2-1’-st image verification unit 128B-1 may verifythe ‘2-2’-nd image in the first direction detected by the ‘2-2-1’-stimage detection unit 126B-1, and output the verified result to themoving object information determination unit 130 via the output terminalOUT 211 (step 438).

After step 446, the ‘2-2-2’-nd image verification unit 128B-2 may verifythe ‘2-2’-nd image in the second direction detected by the ‘2-2-2’-ndimage detection unit 126B-2, and output the verified result to themoving object information determination unit 130 via the output terminalOUT 212 (step 448).

In some cases, at least one of the ‘1-2-1’-st, ‘1-2-2’-nd, ‘2-2-1’-st,or ‘2-2-2’-nd image verification unit 128A-1, 128A-2, 128B-1, or 128B-2may be omitted.

FIG. 10 is a block diagram of another embodiment 120C of the imageextraction unit 120 shown in FIG. 1, which may include first and secondcommon image detection units 121A and 121B, first and second imageclassification units 123A and 123B, ‘1-1-1’-st and ‘1-1-2’-nd imageverification units 124A-1 and 124A-2, ‘1-2-1’-st and ‘1-2-2’-nd imagedetection units 126A-1 and 126A-2, ‘1-2-1’-st and ‘1-2-2’-nd imageverification units 128A-1 and 128A-2, ‘2-1-1’-st and the ‘2-1-2’-ndimage verification units 124B-1, and 124B-2, ‘2-2-1’-st and ‘2-2-2’-ndimage detection units 126B-1 and 126B-2, and ‘2-2-1’-st and ‘2-2-2’-ndimage verification units 128B-1 and 128B-2.

FIG. 11 is a flowchart illustrating another embodiment 220C of step 220shown in FIG. 2.

Hereinafter, step 220C shown in FIG. 11 is described as being performedby the image extraction unit 120C shown in FIG. 10, but embodiments arenot limited thereto. That is, step 220C shown in FIG. 11 may beperformed by the image extraction unit 120 having a differentconfiguration from the image extraction unit 120C shown in FIG. 10.While the image extraction unit 120C shown in FIG. 10 is described asoperating in the order shown in FIG. 11, embodiments are not limitedthereto. That is, according to another embodiment, the image extractionunit 120C shown in FIG. 10 may operate in a different order from themethod 220C shown in FIG. 11.

First, it is determined whether an peripheral image acquired by theimage acquisition unit 110 includes a ‘2-1’-st image (step 502). If theperipheral image includes the ‘2-1’-st image, the second common imagedetection unit 121B detects the ‘2-1’-st image from the peripheral imagereceived from the image acquisition unit 110 through an input terminalIN1 (step 510).

Steps 502 and 510 may be performed by the second common image detectionunit 121B for the following reason. If the ‘2-1’-st image of the wholelower body of the moving object is included in the peripheral image, thesecond common image detection unit 121B may detect the ‘2-1’-st image.However, if the ‘2-1’-st image is not included in the peripheral image,the second common image detection unit 121B may not detect the ‘2-1’-stimage.

If the peripheral image does not include the ‘2-1’-st image, it isdetermined whether the peripheral image includes the ‘1-1’-st image(step 504). If the peripheral image includes the ‘1-1’-st image, thefirst common image detection unit 121A detects the ‘1-1’-st image fromthe peripheral image received from the image acquisition unit 110 viathe input terminal IN1 (step 506).

Steps 502, 504, and 506 are performed by the first common imagedetection unit 121A for the following reason. If the ‘1-1’-st image ofthe whole upper body of the moving object is included in the peripheralimage, the first common image detection unit 121A may detect the‘1-1’-st image. However, if the ‘1-1’-st image is not included in theperipheral image, the first common image detection unit 121A does notdetect the ‘1-1’-st image.

While it is illustrated in step 220C shown in FIG. 11 that the firstimage is not detected and only the second image is detected from theperipheral image when both the first image and the second image arecontained in the peripheral image, embodiments are not limited thereto.

According to another embodiment, when both the first image and thesecond image are contained in the peripheral image, only the first imagemay be detected without detecting the second image.

According to another embodiment, when both the first image and thesecond image are contained in the peripheral image, both the first imageand the second image may be detected.

After step 506, it is determined whether the moving object faces in thefirst direction (step 508). If the moving object faces in the firstdirection, the first image classification unit 123A classifies the‘1-1’-st image in the first direction from the ‘1-1’-st image detectedby the first common image detection unit 121A and outputs the classifiedfirst ‘1-1’-st image in the first direction to the ‘1-1-1’-st imageverification unit 124A-1 (step 522). However, if the moving object facesin the second direction, the first image classification unit 123Aclassifies the ‘1-1’-st image in the second direction from the ‘1-1’-stimage detected by the first common image detection unit 121A and outputsthe classified first ‘1-1’-st image in the second direction to the‘1-1-2’-nd image verification unit 124A-2 (step 532). In this way, thefirst image classification unit 123A serves to classify the ‘1-1’-stimage according to the direction in which the moving object faces.

Here, the first and second directions are the same as those describedregarding the image extraction unit 120B and the method 220B of themoving object information providing device for a vehicle according tothe embodiment shown in FIGS. 8 and 9.

Steps 508, 522 and 532 are performed by the first image classificationunit 123A for the following reason. If the moving object contained inthe peripheral image including the ‘1-1’-st image faces in the first orsecond direction, the first image classification unit 123A is capable ofclassifying the moving object according to the direction in which themoving object faces.

In order to perform the operations described above, the first imageclassification unit 123A may pre-store formatted patterns of the‘1-1’-st image in the first direction and the second direction, andcompare the pre-stored patterns with the pattern of the ‘1-1’-st imagedetected by the first common image detection unit 121A to classify the‘1-1’-st image in the first or second direction.

After step 510, it is determined whether the moving object faces in thefirst direction (step 512). If the moving object faces in the firstdirection, the second image classification unit 123B classifies the‘2-1’-st image in the first direction from the ‘2-1’-st image detectedby the second common image detection unit 121B, and outputs the ‘2-1’-stimage in the first direction to the ‘2-1-1’-st image verification unit124B-1 (step 542). On the other hand, if the moving object faces in thesecond direction, the second image classification unit 123B classifiesthe ‘2-1’-st image in the second direction from the ‘2-1’-st imagedetected by the second common image detection unit 121B, and outputs theclassified ‘2-1’-st image in the second direction to the ‘2-1-2’-ndimage verification unit 124B-2 (Step 552). In this way, the second imageclassifier 123B serves to classify the ‘2-1’-st image according to thedirection in which the moving object faces.

Steps 512, 542, and 552 are performed by the second image classifier123B for the following reason. If the moving object contained in theperipheral image including the ‘2-1’-st image faces in the first orsecond direction, the second image classification unit 123B is capableof classifying the moving object according to the direction in which themoving object faces.

In order to perform the operations described above, the second imageclassification unit 123B may pre-store formatted patterns of the‘2-1’-st image in the first direction and the second direction, andcompare the pre-stored patterns with the pattern of the ‘2-1’-st imagedetected by the second common image detection unit 121B to classify the‘2-1’-st image in the first or second direction.

The ‘1-1-1’-st and ‘1-1-2’-nd image verification units 124A-1 and124A-2, the ‘1-2-1’-st and ‘1-2-2’-nd image detection units 126A-1 and126A-2, the ‘1-2-1’-st and ‘1-2-2’-nd image verification units 128A-1and 128A-2, the ‘2-1-1’-st and ‘2-1-2’-nd image verification units124B-1 and 124B-2, the ‘2-2-1’-st and ‘2-2-2’-nd image detection units126B-1 and 126B-2, and the ‘2-2-1’-st and ‘2-2-2’-nd image verificationunits 128B-1 and 128B-2 shown in FIG. 10 are identical to the ‘1-1-1’-stand ‘1-1-2’-nd image verification units 124A-1 and 124A-2, the‘1-2-1’-st and d image detection units 126A-1 and 126A-2, the ‘1-2-1’-stand ‘1-2-2’-nd image verification units 128A-1 and 128A-2, the‘2-1-1’-st and ‘2-1-2’-nd image verification units 124B-1 and 124B-2,the ‘2-2-1’-st and ‘2-2-2’-nd image detection units 126B-1 and 126B-2,and the ‘2-2-1’-st and ‘2-2-2’-nd image verification units 128B-1 and128B-2 shown in FIG. 8, respectively, and perform the same functions.Therefore, they are assigned the same reference numerals and a redundantdescription thereof is omitted.

Steps 524, 526, 528, 534, 536, 538, 544, 546, 548, 554, 556 and 558shown in FIG. 11 are the same as steps 414, 416, 418, 424, 426, 428,434, 436, 438, 444, 446 and 448 shown in FIG. 9, respectively, andtherefore a detailed description thereof is omitted.

That is, after step 522, the ‘1-1-1’-st image verification unit 124A-1may verify the ‘1-1’-st image in the first direction classified by thefirst image classification unit 123A, and output the verified result tothe ‘1-2-1’-st image detection unit 126A-1 (step 524). After step 542,the ‘2-1-1’-st image verification unit 124B-1 may verify the ‘2-1’-stimage in the first direction classified by the second imageclassification unit 123B, and output the verified result to the‘2-2-1’-st image detection unit 126B-1 (step 544).

After step 532, the ‘1-1-2’-nd image verification unit 124A-2 may verifythe ‘1-1’-st image in the second direction detected by the first imageclassification unit 123A, and output the verified result to the‘1-2-2’-nd image detection unit 126A-2 (step 534). After step 552, the‘2-1-2’-nd image verification unit 124B-2 may verify the ‘2-1’-st imagein the second direction detected by the second image classification unit123B, and output the verified result to the ‘2-2-2’-nd image detectionunit 126B-2 (step 554).

In the case of the embodiment shown in FIGS. 8 and 9, in order to detectthe ‘1-1’-st image in the first or second direction or the ‘2-1’-stimage in the first or second direction, the first pattern of the‘1-1’-st image in the first or second direction or the second pattern ofthe ‘2-1’-st image in the first or second direction is compared with thewhole pattern of the peripheral image, and therefore the amount ofcomputation is increased. On the other hand, in the case of theembodiment shown in FIGS. 10 and 11, the ‘1-1’-st or ‘2-1’-st image isdetected from the peripheral image, and then the first pattern of the‘1-1’-st image in the first or second direction or the second pattern ofthe ‘2-1’-st image in the first or second direction is compared with thepattern of the detected ‘1-1’-st or ‘2-1’-st image instead of the wholepattern of the peripheral image. Therefore, the amount of computationmay be reduced. In addition, in terms of the time required to providethe first or second image to the moving object information determinationunit 130 as well as the computation amount, the embodiment shown inFIGS. 10 and 11 may be more advantageous than the embodiment shown inFIGS. 8 and 9.

While it is described in the embodiments shown in FIGS. 8 to 11 thatthere are two directions in which the moving object faces, which are thefirst and second directions, embodiments are not limited thereto. Thatis, the direction in which the moving object faces may vary. Forexample, the second direction different from the first direction inwhich the moving object faces the front of the vehicle as describedabove may include a ‘2-1’-st direction, in which the moving object facesaway from the vehicle, a ‘2-2’-nd direction in which the right side ofthe moving object faces the vehicle, and a ‘2-3’-rd direction in whichthe left side of the moving object faces the vehicle.

In the case of FIGS. 8 to 11, since there are two directions in whichthe moving object may face (i.e., the first and second directions), itis illustrated that the number of image detection units for detectingthe ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd images and the number ofimage verification units for verifying each of images are two,respectively. However, when there are four kinds of directions in whichthe moving object faces (that is, the first direction and the ‘2-1’-stto ‘2-3’-rd directions) as described above, the number of imagedetection units for detecting the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and‘2-2’-nd images and the number of image verification units for verifyingeach of images may be four, respectively.

FIG. 12 is a block diagram of another embodiment 120D of the imageextraction unit 120 shown in FIG. 1, which may include ‘1-2’-nd and‘2-2’-nd image detection units 126C and 126D, ‘1-2’-nd and ‘2-2’-ndimage verification units 128C and 128D, first and second imagedetermination units 122C and 122D, and ‘1-1’-st and ‘2-1’-st imageverification units 124C and 124D.

FIG. 13 is a flowchart illustrating another embodiment 220D of step 220shown in FIG. 2.

Hereinafter, step 220D shown in FIG. 13 is described as being performedby the image extraction unit 120D shown in FIG. 12, but embodiments arenot limited thereto. That is, step 220D shown in FIG. 13 may beperformed by the image extraction unit 120 having a differentconfiguration from the image extraction unit 120D. In addition, theimage extraction unit 120D shown in FIG. 12 is described as operating inthe order shown in FIG. 13, but embodiments are not limited thereto.That is, according to another embodiment, the image extraction unit 120Dshown in FIG. 12 may operate in a different order from the method 220Dshown in FIG. 13.

First, it is determined whether an peripheral image acquired by theimage acquisition unit 110 includes a ‘2-2’-nd image (step 602). If theperipheral image includes the ‘2-2’-nd image, the ‘2-2’-nd imagedetection unit 126D detects the ‘2-2’-nd image from the peripheral imagereceived from the image acquisition unit 110 via the input terminal IN1(step 614).

Steps 602 and 614 are performed in the ‘2-2’-nd image detection unit126D for the following reason. If the ‘2-2’-nd image is included in theperipheral image, the ‘2-2’-nd image detection unit 126D may detect the‘2-2’-nd image. However, if the ‘2-2’-nd image is not included in theperipheral image, the ‘2-2’-nd image detection unit 126D does not detectthe ‘2-2’-nd image.

If the peripheral image does not include the ‘2-2’-nd image, it isdetermined whether the peripheral image includes the ‘1-2’-nd image(step 604). If the peripheral image includes the ‘1-2’-nd image, the‘1-2’-nd image detection unit 126C detects the ‘1-2’-nd image from theperipheral image received from the image acquisition unit 110 via theinput terminal IN1 (step 606).

Steps 602, 604, and 606 are performed by the ‘1-2’-nd image detectionunit 126C for the following reason. If the ‘1-2’-nd image is included inthe peripheral image, the ‘1-2’-nd image detection unit 126C may detectthe ‘1-2’-nd image. If the ‘1-2’-nd image is not included in theperipheral image, the ‘1-2’-nd image detection unit 126C does not detectthe ‘1-2’-nd image.

Depending on the situation around the vehicle, the peripheral image mayinclude both the ‘1-2’-nd image and the ‘2-2’-nd image. That is, animage of a part of the lower body of the moving object, for example, the‘2-2’-nd image, which is the image of the feet, and an image of a partof the upper body, for example, the ‘1-2’-nd image, which is the imageof the head may be included in the peripheral image. In the case whereboth the first image and the second image are contained in theperipheral image as described above, according to the 220D operationshown in FIG. 13, only the ‘2-2’-nd image is detected without detectingthe ‘1-2’-nd image. However, embodiments are not limited thereto.

According to another embodiment, when both the ‘1-2’-nd and the ‘2-2’-ndimages are contained in the peripheral image, only the ‘1-2’-nd imagemay be detected without detecting the ‘2-2’-nd image.

According to another embodiment, when both the ‘1-1’-st and ‘2-2’-ndimages are contained in the peripheral image, the ‘1-2’-nd and ‘2-2’-ndimages may be detected.

After step 606, the ‘1-2’-nd image verification unit 128C may verify the‘1-2’-nd image detected by the ‘1-2’-nd image detection unit 126C, andoutput the verified result to the first video determination unit 122C(step 608).

After step 614, the ‘2-2’-nd image verification unit 128D may verify the‘2-2’-nd image detected by the ‘2-2’-nd image detection unit 126D, andoutput the verified result to the second image determination unit 122D(step 616).

After step 608, the first image determination unit 122C determines the‘1-1’-st image from the ‘1-2’-nd image verified by the ‘1-2’-nd imageverification unit 128C (step 610). For example, as illustrated in FIG.4B, if the 1-2 image is the image of the head in the upper body of themoving object, the ‘1-2’-nd image I12 of the head of the moving objectMB may be verified, and then the first image determination unit 122C maydetermine that the ‘1-1’-st image I11 of the upper body exists at thelower end of the head. While the ‘1-1’-st image I11 of the upper body isillustrated as including the ‘1-2’-nd image I12 of the head, embodimentsare not limited thereto.

After step 616, the second image determination unit 122D determines the‘2-1’-st image from the ‘2-2’-nd image verified by the ‘2-2’-nd imageverification unit 128D (step 618). For example, as illustrated in FIG.3B or 5B, when the ‘2-2’-nd image is an image of the feet of the movingobject, the ‘2-2’-nd images I22-1 and I22-2 of the feet may be verified,and then it may be determined that the ‘2-1’-st image I21 of the lowerbody exists at the upper end of the feet. While the ‘2-1’-st image I21of the lower body is illustrated as including the ‘2-2’-nd images I22-1and I22-2 of the feet, embodiments are not limited thereto.

In some cases, at least one of the ‘1-2’-nd image verification unit 128Cor the ‘2-2’-nd image verification unit 128D shown in FIG. 12 and atleast one of step 608 or step 616 shown in FIG. 13 may be omitted.

If the ‘1-2’-nd image verification unit 128C and step 608 are omitted,the first image determination unit 122C determines the ‘1-1’-st imagefrom the ‘1-2’-nd image detected by the ‘1-1’-st image detection unit126C after step 608 (step 610). If the ‘2-2’-nd image verification unit128D and step 616 are omitted, the second image determination unit 122Ddetermines the ‘2-1’-st image from the ‘2-2’-nd image detected by the‘2-2’ image detection unit 126D (step 618).

According to another embodiment, the first image determination unit122C, the first image verification unit 124C, the second imagedetermination unit 122D, and the ‘2-1’-st image verification unit 124Dshown in FIG. 12 may be omitted, and steps 610, 612, 618, and 620 shownin FIG. 13 may be omitted. In this case, the ‘1-2’-nd image verified bythe ‘1-2’-nd image verification unit 128C may be output to the movingobject information determination unit 130 via the output terminal OUT12,and the ‘2-2’-nd image verified by the ‘2-2’-nd image verification unit128D may be output to the moving object information determination unit130 via the output terminal OUT22. Therefore, the image extraction unit120D does not provide the ‘1-1’-st and ‘2-1’-st images but provides atleast one of the ‘1-2’-nd or ‘2-2’-nd image to the moving imageinformation determination unit 130.

After step 610, the ‘1-1’-st image verification unit 124C may verify the‘1-1’-st image determined by the first image determination unit 122C andoutput the verified result to the moving object informationdetermination unit 130 via the output terminal OUT12 (step 612). Afterstep 618, the ‘2-1’-st image verification unit 124D may verify the‘2-1’-st image determined by the second image determination unit 122D,and output the verified result to the moving object informationdetermination unit 130 via the output terminal OUT22 (step 620).

In some cases, at least one of the ‘1-1’-st image verification unit 124Cor the ‘2-1’-st image verification unit 124D shown in FIG. 12 may beomitted, and at least one of step 612 or step 620 shown in FIG. 13 maybe omitted.

Each of the ‘1-1’-st image detection unit 122A, the ‘1-2’-nd imagedetection units 126A and 126C, the ‘2-1’-st image detection unit 122B,the ‘2-2’-nd image detection units 126B and 126D, the ‘1-1-1’-st imagedetection unit 122A-1, the ‘1-2-1’-st image detection unit 126A-1, the‘1-1-2’-nd image detection unit 122A-2, the ‘1-2-2’-nd image detectionunit 126A-2, the ‘2-2-1’-st image detection unit 122B-1, the ‘2-2-1’-stimage detection unit 126B-1, the ‘2-1-2’-nd image detection unit 122B-2,the ‘2-2-2’-nd image detection unit 126B-2, the first and second commonimage detection units 121A and 121B and the first and second imageclassification units 123A and 123B described above may detect orclassify a corresponding one of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and‘2-2’-nd images using a various method. For example, each of the‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd images may be detected using amoving object detection approach, an appearance-based approach, or acontour-based approach. Here, the moving object detection approachincludes, for example, background subtraction, temporal differencing,and optical flow. The appearance-based approach includes, for example,Viola-Jones, HOG (Histogram of Oriented Gradients)-SVM (Support VectorMachine), Principal Component Analysis (PCA), Linear DiscriminantAnalysis (LDA), Independent Component Analysis (ICA) and Non-NegativeMatrix Factorization (NMF). Here, Viola-Jones includes, for example,Haarr/HOG/LBP. In addition, the contour-based approach includes, forexample, active contour or chamfer matching.

Each of the ‘1-1’-st image verification units 124A and 124C, the‘2-1’-st image verification units 124B and 124D, the ‘1-2’-nd imageverification units 128A and 128C, the ‘2-2’-nd image verification units128B and 128D, the ‘1-1-1’-st image verification unit 124A-1, the‘1-1-2’-nd image verification unit 124A-2, the ‘2-1-1’-st imageverification unit 124B-1, the ‘2-1-2’-nd image verification unit 124B-2,the ‘1-2-1’-st image verification unit 128A-1, the ‘1-2-2’-nd imageverification unit 128A-2, the 2-2 1 image verification unit 128B-1 andthe ‘2-2-2’-nd image verification unit 128B-2 may verify a correspondingimage using at least one of the position, size, or pattern of the movingobject.

FIG. 14 exemplarily shows a peripheral image PI for explainingverification of a detected image.

For example, referring to FIG. 14, the peripheral image PI acquired bythe image acquisition unit 110 contains a moving object MB walkingaround the vehicle. Here, it is assumed that ‘2-2’-nd images 810 and 812are detected by the ‘2-2’-nd image detection units 126B and 126D. Thesecond ‘2-2’-nd image 812 is correctly detected as an imagecorresponding to the feet of the lower body of the moving object MBwhile the second ‘2-2’-nd image 810 is incorrectly detected as an imagecorresponding to a fire hydrant 830, but the moving object MB.Accordingly, the ‘2-2’-nd image verification units 128B and 128D verifythe ‘2-2’-nd images 810 and 812, and filter out the ‘2-2’-nd image 810corresponding to the fire hydrant 830. In this manner, the imageverification unit serves to filter only the correct image related to themoving object in the detected image. To this end, the image verificationunits may use at least one of the position, size, or pattern of themoving object.

FIG. 15 is a block diagram of an embodiment of the respective imageverification units shown in FIGS. 6, 8, 10, and 12, which may include athird image verification unit 702 and a fourth image verification unit704.

Each of the ‘1-1’-st image verification units 124A and 124C, the‘2-1’-st image verification units 124B and 124D, the ‘1-2’-nd imageverification units 128A and 128C and the ‘2-2’-nd image verificationunits 128B and 128D, the ‘1-1-1’-st image verification unit 124A-1, the‘1-1-2’-nd image verification unit 124A-2, the ‘2-1-1’-st imageverification unit 124B-1, the ‘2-1-2’-nd image verification unit 124B-2,the ‘1-2-1’-st image verification unit 128A-1, the ‘1-2-2’-nd imageverification unit 128A-2, the ‘2-2-1’-st image verification unit 128B-1,and the ‘2-2-2’-nd image verification unit 128B-2 shown in FIGS. 6, 8,10 and 12 may be implemented as shown in FIG. 15.

The third image verification unit 702 may receive the ‘1-1’-st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image via the input terminal IN2, andverify the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image using theposition and size of the moving object.

In addition, the fourth image verification unit 704 may receive the‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image, recognize the patternof the moving object, and verify ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or‘2-2’-nd image using the recognized result. To this end, the ‘1-1’-st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image may be bypassed by the third imageverification unit 702 and provided to the fourth image verification unit704, or may be directly provided from the input terminal IN2 to thefourth image verification unit 704 as shown in FIG. 15.

To this end, the fourth image verification unit 704 may be implementedby a combination of various feature extraction methods and classifiers.For example, HOG, LBP, the Scale Invariant Feature Descriptor (SIFT),the Gabor filter, PCA, ICA, and NMF are examples of the featureextraction methods. Examples of the classifiers include SVM, decisiontrees, random forests, or neural networks.

Further, the image may be verified using a method other than theabove-described approaches for verifying the image. This is intended toincrease accuracy of verification. For example, if PCA is used to verifythe detected image, the image may be verified with accuracy higher thanwhen methods other than PCA, such as NMF, are used to verify the image.

The ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image verification unit124A, 124B, 124C, 124D, 124A-1, 124A-2, 124B-1, 124B-2, 128A, 128B,128C, 128D, 128A-1, 128A-2, 128B-1, or 128B-2 may include third andfourth image verification units 702 and 704 illustrated in FIG. 15. Thefourth image verification unit 704 may re-verify the verified ‘1-1’-st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image verified by the third imageverification unit 702.

According to another embodiment, the third image verification unit 702may re-verify the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd imageverified by the fourth image verification unit 704.

According to yet another embodiment, the image verification unit 124A,124B, 124C, 124D, 124A-1, 124A-2, 124B-1, 124B-2, 128A, 128B, 128C,128D, 128A-1, 128A-2, 128B-1 or 128B-2 may include only one of the thirdimage verification unit 702 and the fourth image verification unit 704.

FIG. 16 shows an exemplary peripheral image PI for explaining adetection area A1 and an alert area A2.

In the image extraction unit 120, 120A, 120B, 120C, 120D, the imagedetection unit 122A, 122B, 122A-1, 122A-2, 122B-1, 122B-2, 126A, 126A-1,126A-2, 126B, 126B-1, 126B-2, 126C, 126D may detect at least one of thefirst or second image from the detection area of the acquired peripheralimage PI. For example, referring to FIG. 16, the detection area A1 maybe an area covering points within a first distance from the vehicle. Forexample, the first distance may be 5 meters, but embodiments are notlimited thereto. Therefore, for a moving object in the non-detectionarea outside the detection area A1, the first and second images may notbe detected.

Referring back to FIGS. 1 and 2, after step 220, the moving objectinformation determination unit 130 may determine (or, generate) movingobject information using at least one of the first image or the secondimage extracted by the image extraction unit 120 (step 230). Here, the“moving object information” is information on the moving object and mayinclude any information that may reduce or prevent accidents caused bythe vehicle when recognized by the driver or the passenger of thevehicle. For example, the moving object information may include at leastone of information about whether or not a moving object is present inthe vicinity of the vehicle or the position of the moving object.

The moving object information determination unit 130 may use at leastone of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image to determinethe moving object information. For example, the moving objectinformation determination unit 130 may determine the moving objectinformation using a combination of multiple ones of the ‘1-1’-st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd images.

For example, when the upper body of the moving object is hidden, themoving object information determination unit 130 may correctly determinethe position of the moving object by combining the ‘2-1’-st image andthe ‘2-2’-nd image. Alternatively, when the lower body of the movingobject is hidden, the moving object information determination unit 130may correctly determine the position of the moving object by combiningthe ‘1-1’-st image and the ‘1-2’-nd image. Alternatively, when a largepart of the lower body of the moving object is hidden and a large partof the upper body of the moving object is hidden, the combination of the‘1-1’-st image and the ‘1-2’-nd image and the combination of the‘2-1’-st image and the ‘2-2’-nd image may be used to more correctlydetermine the moving object information. Therefore, even when at least apart of the upper body or the lower body of the moving object is hidden,the moving object information may be correctly determined.

According to another embodiment, the moving object informationdetermination unit 130 may determine the moving object information usingonly at least one of the ‘1-2’-nd or ‘2-2’-nd image or using only atleast one of the ‘1-1’-nd or ‘2-1’-nd image.

In addition, when the image extraction unit 120 extracts both the firstimage and the second image, the moving object information determinationunit 130 may determine the moving object information using an imageextracted first among the first image and the second image. However,embodiments are not limited thereto.

Alternatively, when the image extraction unit 120 extracts both thefirst image and the second image as described above, the moving objectinformation determination unit 130 may determine the moving objectinformation using only the second image. This may be intended tofacilitate tracking of the position of the moving object, as will bedescribed later, since the moving object is moved by the lower bodyrather than the upper body and since the upper body of the moving objectis more likely to be hidden than the lower body is. As described above,when the second image of the lower body rather than the upper body ofthe moving object is used, the moving object information may bedetermined more accurately and quickly. However, embodiments are notlimited thereto.

When the peripheral image includes neither the ‘1-1’ image nor the‘2-1’-st image in FIGS. 7, 9 and 11, or when the peripheral imageincludes neither the ‘1-2’-nd image nor the ‘2-2’-nd image in FIG. 13,the first image or the second image is not provided from the imageextraction unit 120 to the moving object information determination unit130. In this case, the moving object information determination unit 130may determine that the moving object does not exist.

According to an embodiment, the moving object information providingdevice 100 for a vehicle may further include a moving object alert unit140. After step 230, the moving object alert unit 140 may announce themoving object information to a person who needs to be warned to preventa traffic accident, such as a driver of the vehicle or a driver ofanother vehicle located in the vicinity of the vehicle, in response tothe result determined by the moving object information determinationunit 130 (step 240). Here, the announced moving object information maybe, for example, at least one of the presence or position of the movingobject.

According to an embodiment, when the moving object enters the alertarea, the moving object alert unit 140 may announce the moving objectinformation. Here, the alert area may be an area of points within asecond distance from the vehicle. For example, referring to FIG. 16, thealert area A2 may be an area closer to the vehicle than the detectionarea A1. For example, the second distance may be 3 meters. However,embodiments are not limited thereto. When the moving object is locatedoutside the alert area A2, the moving object alert unit 140 may notannounce the moving object information, but embodiments are not limitedthereto.

FIG. 17 is a block diagram of an embodiment 130A of the moving objectinformation determination unit 130 shown in FIG. 1, which may include aposition tracking unit 132 and an information generation unit 134.

Referring to FIG. 17, the position tracking unit 132 may track theposition of the moving object using at least one of the first or secondimage extracted by the image extraction unit 120, and output the trackedposition via the output terminal OUT4. The tracked position may beoutput to the information generating unit 134 or to the moving objectalert unit 140. The moving object alert unit 140 may announce the movingobject information according to the tracked position of the movingobject. For example, the position tracking unit 132 may receive thefirst image and the second image extracted by the image extraction unit120 via the input terminals IN3 and IN4, respectively.

In addition, the position tracking unit 132 may determine that a movingobject exists at a position where a combination of the ‘2-1’-st imageand the ‘2-2’-nd image is confirmed. For example, referring to FIGS. 3Band 5B, it is determined that a moving object MB exists at a positionwhere the combination of the ‘2-1’-st image I21 and the ‘2-2’-nd imagesI22-1 and I22-2 is confirmed.

Alternatively, the position tracking unit 132 may determine that amoving object exists at a position where a combination of the ‘1-1’-stimage and the ‘1-2’-nd image is confirmed. For example, referring toFIG. 4B, it may be determined that a moving object MB exists at aposition where the combination of the ‘1-1’-st image I11 and the‘1-2’-nd image I12 is confirmed.

For example, when the first or second image provided from the imageextraction unit 120 is a previously detected image, the path of movementof the moving object may be tracked by integrating the temporaldetection results. For example, the position tracking unit 132 may trackthe position of the moving object by associating the continuouslydetected sizes of the moving object with the continuously detectedpositions of the moving object based on similarity of the continuouslydetected sizes and the continuously detected positions.

Alternatively, when the moving object is intermittently detected, theposition tracking unit 132 may detect the moving object using variousvisual tracking methods such as using the similarity of colors,gradients or shapes of the moving object.

In addition, the position tracking unit 132 may predict and correct theresult of tracking the position of the moving object by variousfiltering techniques such as stochastic filtering. Such filteringtechniques include, for example, Kalman filtering, extended Kalmanfiltering, unscented Kalman filtering, or particle filter.

The information generating unit 134 may generate (or, determine) movingobject information using at least one of the first image or the secondimage extracted by the image extraction unit 120 or the tracked positionoutput from the position tracking unit 132, and output the generatedinformation via an output terminal OUT5.

As a result, the moving object information providing device and methodfor a vehicle according to the embodiments may determine and provideinformation on the moving object using at least one of the ‘1-1’-st,‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image, thereby helping to preventvehicles from causing traffic accidents.

Further, as shown in FIGS. 3A and 3B, when the moving object MB isstanding close to the vehicle, the upper body thereof may be hidden.When the moving object is squatting as shown in FIGS. 4A and 4B, thelower body thereof may be hidden. According to the conventional methodof detecting the whole body, the information on the moving object maynot be correctly determined due to various postures of the movingobject. In contrast, the moving object information providing deviceaccording to an embodiment divides a peripheral image into a first imageof the upper body of the moving object and a second image of the lowerbody of the moving object, and determines the information on the movingobject using the divided images. Therefore, even in a situation wherethe lower body or the upper body of the moving object is hidden, theinformation on the moving object may be correctly determined, and theinfluence of distortion of the fisheye lens may be reduced.

Further, in the moving object information providing method according toan embodiment, the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd images aredetected and verified, thereby enhancing the accuracy of the ‘1-1’-st,‘1-2’-nd, ‘2-1’-st and ‘2-2’-nd images. Therefore, the information onthe moving object may be determined more accurately.

In addition, the ‘2-2’-nd image of a part of the lower body of themoving object is used to determine the moving object information, it maybe easy to determine whether or not the moving object is positioned inthe alert area A2.

Further, the moving object information may be determined using only the‘1-2’-nd image or only the ‘1-1’-st image, not the combination of the‘1-1’-st image and the ‘1-2’-nd image. In addition, the information onthe moving object may be determined using only the ‘2-1’-st image oronly the ‘2-2’-nd image, not the combination of the ‘2-1’-st image andthe ‘2-2’-nd image. Therefore, the structure of the moving objectinformation providing device for a vehicle may be simplified, and themoving object information providing method for a vehicle may be executedquickly and easily.

In addition, the moving object information providing method and deviceaccording to an embodiment may determine the moving object informationby distinguishing between the lower body and the upper body of themoving object. Accordingly, the moving object information may beaccurately determined using the second image of the lower body of themoving object when the lower body of the moving object is dynamicallychanged without dynamic change of the upper body. Alternatively, themoving object information may be accurately determined using the firstimage of the upper body of the moving object when only the upper body isdynamically changed without dynamic change of the lower body. Moreover,since the moving object information is determined using the second imageof the lower body of the moving object when the upper body of the movingobject is hidden and using the first image of the upper body of themoving object when the lower body of the moving object is hidden, themoving object information may be accurately determined even when theupper body or the lower body of the moving object is hidden. Inaddition, the moving object information may be determined using only theimage of the head, which is a part of the upper body, or the feet, whichis a part of the lower body when most of the upper and lower bodies ofthe moving object are hidden. Therefore, the moving object informationmay be accurately determined even when most of the upper and lowerbodies of the moving object are hidden.

In addition, the moving object information may be determined moreaccurately using both the result obtained by combining the ‘1-1’-stimage and the ‘1-2’-nd image and the result obtained by combining the‘2-1’-st image and the ‘2-2’-nd image.

As such, since moving object information is determined using at leastone of the ‘1-1’-st, ‘1-2’-nd, ‘2-1’-st, or ‘2-2’-nd image, theinformation on the moving object may be accurately determined andannounced to the driver in various situations including the standingposition, the position very close to the vehicle, and squatting positionof the moving object located near the vehicle and the small size of themoving object. Thereby, traffic accidents may be prevented.

A recording medium on which a program for executing the moving objectinformation providing method 200 for a vehicle performed by the movingobject information providing device 100 is recorded may have a programfor implementing a function of acquiring a peripheral image of thevehicle, a function of extracting from the acquired peripheral image atleast one of a first image of the upper body of the moving object Or asecond image of the lower body of the moving object and a function ofdetermining at least one of the presence/absence or position of themoving object using at least one of the extracted first or second image,and the computer may read the recording medium.

On the computer-readable recording medium, a program having the firstimage including at least one of a ‘1-1’-st image of the whole upper bodyof the moving object or a ‘1-2’-nd image of a part of the upper body ofthe moving object and the second including at least one of a ‘2-1’-stimage of the whole lower body of the moving object or a ‘2-2’-nd imageof a part of the lower body of the moving object may be recorded.

The program for implementing the function of extracting at least one ofthe first image or the second image, which includes a function ofdetecting the ‘1-1’-st image from the acquired peripheral image when theacquired peripheral image includes the ‘1-1’-st image, a function ofdetecting a ‘2-1’-st image from the acquired peripheral image when theacquired peripheral image includes the ‘2-1’-st image, a function ofverifying the detected ‘1-1’-st or ‘2-1’-st image, a function ofdetecting a ‘1-2’-nd or ‘2-2’-nd image from the verified ‘1-1’-st or‘2-1’-st image, respectively, and a function of verifying the detected‘1-2’-nd or ‘2-2’-nd image, may be recorded on a recording medium, andthe computer may read the recording medium.

In addition, the function of detecting the ‘1-1’-st image may include afunction of detecting, when a moving object contained in the ‘1-1’-stimage included in the acquired peripheral image faces in the firstdirection with respect to the vehicle, the ‘1-1’-st image in the firstdirection from the acquired peripheral image and a function ofdetecting, when the moving object contained in the ‘2-1’-st imageincluded in the acquired peripheral image faces in the second directiondifferent from the first direction with respect to the vehicle, the‘1-1’-st image in the second direction from the acquired peripheralimage.

In addition, the program for implementing the function of detecting the‘2-1’-st image, which may include a function of detecting, when a movingobject contained in the ‘2-1’-st image included in the acquiredperipheral image faces in the first direction with respect to thevehicle, the ‘2-1’-st image in the first direction from the acquiredperipheral image and a function of detecting, when the moving objectcontained in the ‘2-1’-st image included in the acquired peripheralimage faces in the second direction with respect to the vehicle, the‘2-1’-st image in the second direction from the acquired peripheralimage, may be recorded on a recording medium and the computer may readthe recording medium.

The program for implementing the function of detecting the 1-1 or‘2-1’-st image may include a function of detecting the ‘1-1’-st imagefrom the acquired peripheral image when the ‘1-1’-st image is includedin the acquired peripheral image, a function of classifying the ‘1-1’-stimage in the first direction from the detected ‘1-1’-st image when themoving object faces in the first direction with respect to the vehiclein the detected ‘1-1’-st image, and a function of classifying the‘1-1’-st image in the second direction from the detected ‘1-1’-st imagewhen the moving object contained in the detected ‘1′1’-st image faces inthe second direction with respect to the vehicle. Alternatively, aprogram for implementing a function of detecting the ‘2-1’-st image fromthe acquired peripheral image when the ‘2-1’-st image is included in theacquired peripheral image, a function of classifying the ‘2-1’-st imagein the first direction from the detected ‘2-1’-st image when the movingobject faces in the first direction with respect to the vehicle in thedetected ‘2-1’-st image, and a function of classifying the ‘2-1’-stimage in the second direction from the detected ‘2-1’-st image when themoving object contained in the detected ‘2-1’-st image faces in thesecond direction with respect to the vehicle may be recorded, and thecomputer may read the recording medium.

The computer-readable recording medium may include all kinds of storagedevices in which data that may be read by a computer system is stored.Examples of the computer-readable recording medium include ROM, RAM,CD-ROM, magnetic tapes, floppy disks, and optical data storage devices,and also include carrier-wave type implementation (e.g., transmissionover the Internet). The computer-readable recording medium may bedistributed to a computer system connected over a network, andcomputer-readable code may be stored and executed thereon in adistributed manner. Functional programs, codes, and code segments forimplementing the method of opening the vehicle trunk may be easilyinferred by programmers in the art to which the embodiments pertain.

While the present disclosure has been particularly shown and describedwith reference to exemplary embodiments thereof, it is to be understoodthat the disclosure is not limited to the disclosed embodiments. It willbe understood by those skilled in the art that various modifications andapplications are possible without departing from the essential featuresof the embodiments. For example, each component specifically shown inthe embodiments may be modified and implemented. It is to be understoodthat all changes and modifications that come within the meaning andrange of equivalency of the claims are therefore intended to be embracedtherein.

MODE FOR INVENTION

The mode for carrying out the disclosure has been fully described in“Best Mode”.

INDUSTRIAL APPLICABILITY

A moving object information providing device and method for a vehicleand a recording medium on which a program for executing the method isrecorded according to an embodiment may be applied to vehicles.

The invention claimed is:
 1. A moving object information providingdevice for a vehicle, comprising: a camera configured to acquire aperipheral image of the vehicle; a processor configured to: extract atleast one of a first image of an upper body of at least one movingobject or a second image of a lower body of the moving object from theperipheral image of the vehicle; and determine as a moving objectinformation at least one of presence/absence of the moving object aroundthe vehicle or a position of the moving object using at least one of theextracted first image or second image, wherein the processor extracts,as the first image, at least one of a ‘1-1’-st image of an entirety ofthe upper body of the moving object; or a ‘1-2’-nd image of a part ofthe upper body of the moving object, and extract, as the second image,at least one of a ‘2-1’-st image of an entirety of the lower body of themoving object; or a ‘2-2’-nd image of a part of the lower body of themoving object, and wherein the processor determines the moving objectinformation using only the second image without using the first image,when the processor extracts the first and second images.
 2. The deviceaccording to claim 1, wherein the processor is further configured to:detect the ‘1-1’-st image from the acquired peripheral image; and detectthe ‘2-1’-st image from the acquired peripheral image.
 3. The deviceaccording to claim 2, wherein the processor is further configured to:verify the ‘1-1’-st image detected by the ‘1-1’-st image detection unit;or verify the ‘2-1’-st image detected by the ‘2-1’-st image detectionunit.
 4. The device according to claim 3, wherein the processor isfurther configured to: detect the ‘1-2’-nd image from the ‘1-1’-stimage; and detect the ‘2-2’-nd image from the ‘2-1’-st image.
 5. Thedevice according to claim 4, wherein the processor is further configuredto: verify the ‘1-2’-nd image; or verify the ‘2-2’-nd image.
 6. Thedevice according to claim 5, wherein the processor is further configuredto: detect the ‘1-1’-st image in a first direction in which the movingobject faces the vehicle from the acquired peripheral image; detect the‘1-1’-st image in a second direction different from the first directionfrom the acquired peripheral image; detect the ‘2-1’-st image in thefirst direction from the acquired peripheral image; and detect the‘2-1’-st image in the second direction from the acquired peripheralimage.
 7. The device according to claim 5, wherein the processor isfurther configured to: detect the ‘1-1’-st image from the acquiredperipheral image; classify the ‘1-1’-st image according to a directionin which the moving object faces with respect to the vehicle from the‘1-1’-st image; detect the ‘2-1’-st image from the acquired peripheralimage; and classify the ‘2-1’-st image according to a direction in whichthe moving object faces with respect to the vehicle from the ‘2-1’-stimage.
 8. The device according to claim 6, wherein the processor isfurther configured to: verify the ‘1-1’-st image in the first direction;or verify the ‘1-1’-st image in the second direction, and wherein theprocessor is further configured to: verify the ‘2-1’-st image in thefirst direction; or verify the ‘2-1’-st image in the second direction.9. The device according to claim 8, wherein the processor is furtherconfigured to: detect the ‘1-2’-nd image in the first direction from the‘1-1’-st image; detect the ‘1-2’-nd image in the second direction fromthe ‘1-2’-nd image detected by the processor; detect ‘2-2’-nd image inthe first direction from the ‘2-1’-st image detected by the processor;and detect the ‘2-2’-nd image in the second direction from the ‘2-1’-stimage detected by the processor.
 10. The device according to claim 9,wherein the processor is further configured to: verify the ‘1-2’-ndimage; or verify the ‘1-2’-nd image, and wherein the processor isfurther configured to: verify the ‘2-2’-nd image; or verify the ‘2-2’-ndimage.
 11. The device according to claim 1, wherein the processor isfurther configured to: detect the ‘1-2’-nd image from the acquiredperipheral image; and detect the ‘2-2’-nd image from the acquiredperipheral image.
 12. The device according to claim 11, wherein theprocessor is further configured to: verify the ‘1-2’-nd image; or verifythe ‘2-2’-nd image.
 13. The device according to claim 12, wherein theprocessor is further configured to: determine the ‘1-1’-st image fromthe ‘1-2’-nd image; and determine the ‘2-1’-st image from the ‘2-2’-ndimage.
 14. The device according to claim 13, wherein the processor isfurther configured to: verify the ‘1-1’-st image determined by the firstimage determination unit; and verify the ‘2-1’-st image.
 15. A methodfor providing moving object information for a vehicle, the methodcomprising: (a) acquiring a peripheral image of the vehicle; (b)extracting at least one of a first image of an upper body of at leastone moving object or a second image of a lower body of the moving objectfrom the acquired peripheral image; and (c) determining as a movingobject information at least one of presence/absence of the moving objector a position of the moving object using at least one of the extractedfirst image or second image, wherein the first image comprises at leastone of: a ‘1-1’-st image of an entirety of the upper body of the movingobject; or a ‘1-2’-nd image of a part of the upper body of the movingobject, wherein the second image comprises at least one of: a ‘2-1’-stimage of an entirety of the lower body of the moving object; or a‘2-2’-nd image of a part of the lower body of the moving object, andwherein the moving object information is determined using only thesecond image without using the first image in step (c), when the firstand second images are extracted in step (b).
 16. The method according toclaim 15, wherein the step (b) comprises: detecting the ‘1-1’-st imagefrom the acquired peripheral image when the acquired peripheral imagecomprises the ‘1-1’-st image; detecting the ‘2-1’-st image from theacquired peripheral image when the acquired peripheral image comprisesthe ‘2-1’-st image; verifying the detected ‘1-1’-st or ‘2-1’-st image;detecting the ‘1-2’-nd or ‘2-2’-nd image from the verified or detected‘1-1’-st or ‘2-1’-st image, respectively; and verifying the detected‘1-2’-nd or ‘2-2’-nd image.
 17. The method according to claim 16,wherein the detecting of the ‘1-1’-st image comprises: detecting the‘1-1’-st image of a first direction in which the moving object faces thevehicle from the acquired peripheral image when the moving objectcontained in the ‘1-1’-st image included in the acquired peripheralimage faces in the first direction with respect to the vehicle; anddetecting the ‘1-1’-st image of a second direction different from thefirst direction from the acquired peripheral image when the movingobject contained in the ‘2-1’-st image included in the acquiredperipheral image faces in the second direction with respect to thevehicle, wherein the detecting of the ‘2-1’-st image comprises:detecting the ‘2-1’-st image of the first direction from the acquiredperipheral image when the moving object contained in the ‘2-1’-st imageincluded in the acquired peripheral image faces in the first directionwith respect to the vehicle; and detecting the ‘2-1’-st image of thesecond direction from the acquired peripheral image when the movingobject contained in the ‘2-1’-st image included in the acquiredperipheral image faces in the second direction with respect to thevehicle.
 18. The method according to claim 16, wherein the detecting ofthe ‘1-1’-st or ‘2-1’-st image comprises: detecting the ‘1-1’-st imagein the acquired peripheral image when the ‘1-1’-st image is includedfrom the acquired peripheral image; classifying the ‘1-1’-st image of afirst direction from the detected ‘1-1’-st image when the moving objectcontained in the detected ‘1-1’-st image faces in the first directionwith respect to the vehicle; classifying the ‘1-1’-st image of a seconddirection different from the first direction from the detected ‘1-1’-stimage when the moving object contained in the detected ‘1-1’-st imagefaces in the second direction with respect to the vehicle; detecting the‘2-1’-st image from the acquired peripheral image when the ‘2-1’-stimage is included in the acquired peripheral image; classifying the‘2-1’-st image of the first direction from the detected ‘2-1’-st imagewhen the moving object contained in the detected ‘2-1’-st image faces inthe first direction with respect to the vehicle; and classifying the‘1-1’-st image of the second direction different from the firstdirection from the detected ‘2-1’-st image when the moving objectcontained in the detected ‘2-1’-st image faces in the second directionwith respect to the vehicle.
 19. The method according to claim 15,wherein the step (b) comprises: detecting the ‘1-2’-nd image from theacquired peripheral image when the acquired peripheral image comprisesthe ‘1-2’-nd image; detecting the ‘2-2’-nd image from the acquiredperipheral image when the acquired peripheral image comprises the‘2-2’-nd image; verifying the detected ‘1-2’-nd or ‘2-2’-nd image;determining the ‘1-1’-st or ‘2-1’-st image from the verified ‘1-2’-nd or‘2-2’-nd image, respectively; and verifying the determined ‘1-1’-st or‘2-1’-st image.
 20. A non-transitory computer-readable recording mediumon which a program for executing a moving object information providingmethod for a vehicle performed by a moving object information providingdevice for the vehicle is recorded, wherein the program is configured toimplement: (a) a function of acquiring a peripheral image of thevehicle; (b) a function of extracting at least one of a first image ofan upper body of a moving object or a second image of a lower body ofthe moving object from the acquired peripheral image; and (c) a functionof determining as a moving object information at least one of presenceof the moving object or a position of the moving object, using at leastone of the extracted first or second image, and wherein the movingobject information is determined using only the second image withoutusing the first image, when the first and second images are extracted inthe function (b).